Patterns in randomly evolving networks: Idiotypic networks 
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We present a model for the evolution of networks of occupied sites on undirected regular graphs. At 
every iteration step in a parallel update I randomly chosen empty sites are occupied and occupied 
sites having degree outside of a given interval (fi,iu) are set empty. Depending on the influx / and 
the values of both lower threshold and upper threshold of the degree different kinds of behaviour 
can be observed. In certain regimes stable long-living patterns appear. We distinguish two types of 
pattern: static patterns arising on graphs with low connectivity and dynamic patterns found on high 
connectivity graphs. Increasing / patterns become unstable and transitions between almost stable 
patterns, interrupted by disordered phases, occur. For still larger I the lifetime of occupied sites 
becomes very small and network structures are dominated by randomness. We develop methods 
to analyze nature and dynamics of these network patterns, give a statistical description of defects 
and fluctuations around them, and elucidate transitions between different patterns. Results and 
methods presented can be applied to a variety of problems in different flelds and a broad class of 
graphs. Aiming chiefly at the modeling of functional networks of interacting antibodies and B-cells 
of the immune system (idiotypic networks) we focus on a class of graphs constructed by bit-chains. 
The biological relevance of the patterns and possible operational modes of idiotypic networks are 
discussed. 

PACS numbers; 64.60.Ak, 05.10.Ln, 02.70.Lq, 87.18.-h 



structural growth. As recently observed major transi- 
tions in a systems dynamics can even be completely gov- 
erned by preceding transformations in its network struc- 
ture 

In the case of idiotypic networks (INW's) in the im- 
mune system pl| ], dynamics and network evolution are 
driven by a continuous influx of new idiotypes from the 
bone marrow. Data from experimental investigations 
suggest a daily bone marrow production only one order 
smaller than the actual network size ||5|, P2|j23| . A second 
principle governing the dynamics of INW's is a set of 
local rules: idiotypes die out if they are under- or over- 
stimulated, i.e. have too few or too many neighbours. 
As Conways famous 'game of life' |2j] suggests, systems 
governed by such local rules can evolve towards highly 
complex self-organized states [ p5| . 

Starting from a simple set of rules intending to mimic 
the dynamics within idiotypic networks in the immune 
system, we present a cellular automaton based model for 
the evolution of networks of occupied sites on graphs. 
Depending on the major parameter, the bone marrow 
influx, the model exhibits parameter regimes in which the 
system self-organizes into static and dynamic patterns, 
shows transitions between such ordered phases and has a 
parameter range in which it is governed by randomness. 
We investigate these regimes and elucidate connections 
between the overall dynamics and the evolution of the 
network structure. 

The organization of this paper is as follows. In the 
next section we introduce the model, give a brief intro- 
duction to idiotypic networks and outline further possible 
applications of our model. Then, in Sec. [Ill, 



I. INTRODUCTION 

Many complex coupled systems can be modeled as net- 
works: Vertices being associated with the systems ele- 
ments and edges representing interactions between them. 
Metabolic networks |jl|,||, food webs social networks 
Q, and networks in the immune system (for some ap- 
proaches see 1^) are just examples of such systems. Fre- 
quently, although the detailed interactions of the ele- 
ments may be rather complicated, the knowledge about 
the underlying network structure facilitates the under- 
standing of essential features of the system as a whole 

In the absence of detailed knowledge about a networks 
topology previous works tended to assume either a com- 
pletely random link structure or clusters given by 
percolation on lattices | [Tc[ | . The complexity of many sys- 
tems, however, emerges as a consequence of some under- 
lying principle for their evolution, e.g., preferential at- 
tachment i pH , optimization of transportation and com- 
munication pathways [p"2|-p^ extinction of the least pop- 
ulated species |l|,|l|,or of the mere fact that the network 
has randomly evolved in time p^ ]. This, altogether may 
lead to a highly organized network topology. 

Recently, much attention has been devoted to a variety 
of networks which exhibit topological properties different 
from random graphs (for a review see p"8|-p0{). For ob- 
taining a detailed understanding of their architecture it 
has proved useful to trace the dynamics of the networks 



we give an 

overview over the typical evolution of the initial dynam- 
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ics of the population and its connection with changes in 
the network structure. Next, in Sec. IV the stationary 



state and its network organization will be characterized. 
In the fifth section the dependence of the systems qualita- 
tive behaviour on the models main parameter, the influx 
/, will be explored, analytical results for small / derived 
from a microscopic view of the network characterizing 
fluctuations around the steady state be presented and an 
explanation for the systems high / behaviour be given. 
In the final section changes in the variety of stationary 
network patterns for more closely linked base graphs will 
be discussed. 

The aim of this paper is not fine tuning or in detail 
matching of one of the applications, but rather generally 
exploring features of the dynamics caused by the window- 
algorithm and understanding its connection with network 
structures thus evolving. 



II. THE MODEL AND SOME APPLICATIONS 

Let G denote a lattice or, more generally, an undirected 
connected K-regular graph (i.e. a graph, every of whose 
vertices is linked to k neighbours). Let w G G denote the 
vertices or sites and the links between sites {i, j) within G 
be described by an adjacency matrix {™ij}i<i j<|g|- For 
simplicity rriij G {0, 1} and niu = 0. In the following G 
will also be called base graph. A vertex u e G can either 
be occupied or empty, i.e., have occupancy s{v) = 1 or 
s{v) — 0, respectively. Empty sites will be called holes. 
A vertex v with label i has degree dv = X]j=i ™u — 
We propose the following algorithm for an evolution of 
occupied and empty sites on G: 

(i) Throw in / occupied vertices, i.e. select i holes ran- 
domly and let them become occupied. This supply 
of / new occupied vertices will be called influx. 

(ii) Check the neighbourhood of every site v € G. If 
V is occupied and has degree dv greater than <u or 
less than t\ the vertex v will be set empty in the 
next timestep. That is: St(y) = 1 — > st+i{v) = if 
dv < ti or dv > iu- The update of (i) and (ii) will 
be parallel. 

(iii) Iterate (i) and (ii). 

The threshold values ti and are model parameters, 
t\ — corresponds to no lower threshold at all. Thus, 
the main characteristics of the algorithm is a window of 
allowed degrees. Hence we call it window algorithm. The 
graph of all occupied vertices after iteration t is finished 
will be denoted by Ft G G and rit = jFtl be called its 
population. 

Note, that in the first instance unlimited growth of the 
network is prevented by the upper threshold t^. Once a 
stable structure has been established, there are no ver- 
tices violating the minimum degree rule unless vertices 



are taken out because they got too many neighbours by 
the last influx. On the other hand, while <u leads to 
an instantaneous removal of vertices due to the most re- 
cent influx, the lower threshold ti can cause avalanches 
and is thus responsible for a memory of a perturbation 
which may last over several iterations. Such avalanches 
are thought to adjust the network structure. 

These considerations make it also plain that for tu — ti 
and tu = t\ + I no long lasting populations can arise. 
In the first case, only a U— regular graph Tt could be 
formed. Then already a small local reorganization would 
lead to an avalanche extinguishing the whole graph. In 
the second case, every vertex of Tt is critical in a sense 
that it is prone to become removed by a small perturba- 
tion caused by the influx. 

Our main motivation to study such a kind of dynamics 
for the evolution of networks comes from trying to model 
idiotypic networks (INWs). The problem, however, ap- 
pears to be far more general. Two other imaginable ap- 
plications will also be sketched briefly: evolving networks 
of interacting species and, for conceiving a vivid picture 
of the above algorithm, a toy system of organzing coins 
in a box. 



A. Idiotypic networks 

Immune response against a broad class of antigen (bac- 
teria, viruses, ...) is triggered by antibodies and B-cells 
(which carry only one type of antibody on their sur- 
face). Schematically this can be sketched as follows. 
Perchance the antigen encounters a shape- and charge- 
complementary antibody. Antibody and antigen thus 
form a complex which marks the antigen as hostile so 
that it will be removed by another functional group of 
cells (e.g. killer cells). If B-cells, which carry exactly 
this type of antibody on their surface come into contact 
with such an antigen they become stimulated, multiply 
into a clone and finally become 'production units' for this 
specific type of antibody. 

It is the basic idea of idiotypic networks |^ that an- 
tibodies can not only recognize antigen, but also com- 
plementary anti-antibodies (the specificities of both are 
then also called idiotypes). This leads to stimulation of 
the respective B-cells and thus causes a kind of a perma- 
nent self-balanced immune activity independent of hostile 
antigen. 

These mutually recognizing idiotypes build the so- 
called idiotypic network. Its vertices or nodes are rep- 
resented by idiotypes, its links by functional interactions 
between them. INWs are thought to play a role in, e.g., 
the preservation of idiotypic memory pl[ , the preven- 
tion of autoimmunity [ p6| , p7[ , self-non-self discriminiza- 
tion and cancer control. 

The interaction of idiotypes within the network can be 
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described by Lotka-Volterra like dynamics, i.e. 



dxi 




where i is an index labelling different idiotype popula- 
tions, Xi the concentration of idiotype i, 7^ its inverse 
lifetime, / a so-called proliferation function describing 
the stimulaton and death of B-cells and antibody pro- 
duction by stimulated B-cells, and an influx rate of 
new idiotype species from the bone marrow. Since se- 
creted antibodies and those on the surface of B-cells are 
not explicitly distinguished a description of INW dynam- 
ics as by Eq. (|l|) belongs to the category of A-models [D . 
Bone marrow production is thought to be uniformly dis- 
tributed and random, i.e., there is a uniform probability 
for every idiotype that a member of its population is pro- 
duced per unit of time. 

It is a common approach to describe an idiotype or 
vertex v of the INW by a bitstring (wi, ...,Vd), Vi e {0, 1} 
of length d |2^. Reasonable estimates for the number 
of possible idiotypes yield a bit-chain length of the order 
d « 35 Idiotypes interact, if they are complementary. 
This is modelled by introducing 'matching-rules' which 
define when bit-strings are connected in the above sense 
of complementarity. For instance, idiotypes can be said 
to interact if they are nearly exactly complementary or - 
in other words- the respective bit-strings match allowing 
for one deviating position, a mismatch, only. Generally, 
allowing bit-strings with m mismatches to interact, one 

obtains a k = X^fcLo -regular graph G^P'^ whose links 

are described by an adjacency matrix with elements: 



overstimulated on the other hand. Too high concentra- 
tions of antibody lead to each receptor of the B-cell being 
bound to only one antibody. Since cross-linking of sev- 
(1) eral antibodies is required in order to stimulate a B-cell 
Q this leads to suppression of this idiotypes population. 
A more careful analysis yields that it is reasonable to as- 
sume that the proliferation function / is log-bell shaped. 
Then, from the steady state conditions of Eq. (|l|) one 
obtains a rough idea of the maximum and the minimum 
degree, ti and t^. Normally, one matching antibody speci- 
fity is sufhcient to cope with an antigen. Therefore in this 
model t\ — 1 which defines the timescales for each itera- 
tion step. 



(m) J 1 if duiij) < m 
1 ■ — \ 

'■^ I otherwise 



B. Interacting species 

A widely used approach to describe the interactions of 
species (macroscopic organisms, cells, reacting molecules, 
...) is by Lotka-Volterra like dynamics of the type set 
forth in Eq. (|l|) |2^]. The index i now distinguishes dif- 
ferent species, Xi denotes their concentrations, — 7^ > 
the net effect of death and birth rate, my the adjacency 
matrix describing their interaction structure, and / a 
function modeling resource competition and the effect 
of predator-prey relations. Now, a species does not re- 
quire a minimum amount of stimulation but is still going 
to be restrained by the effects of resource competition 
and prey. Thus, ti = and > 0. ^ in this context 
models a kind of mutation rate, which describes the ap- 
pearance of new species. Unlike in the case of the bone 
marrow production, ^ is not uniform, but restricted to 
the neighbourhood of already existing species and also 
much smaller. Though probably hard to make out, G 
and rriij describe the graph of all conceivable species and 
(2) their potential interactions. 



Reaction affinities are the stronger the better the match- 
ing of the corresponding idiotypes is. In this respect, in 
the sequence g[1^ c g''^ c ... C G^j™^ gradually smaller 
affinities are taken into account. 

Sometimes it is also convenient to employ another view 
at the link structure of G^^™'' . In the frame of these bit- 
string models every link is associated with a bit-operation 
L, which, applied to one of the vertices that the link con- 
nects, yields the vertex at its opposite end. For links 
between vertices with ideally complementary bit-chains 
i,j we have j = io(i) {i is the inverse of j), for 'one- 
mismatch-links' j = (i) {i is obtained by inverting j 
and changing the bit at position ki), for 'two-mismatch- 
links' j = Lfcjfe2(?), etc. Thus, in this sense links are also 
labelled. 

Using a discrete version of Eq. (|l]) idiotypes are either 
present {xi = 1) or absent {xi = 0). An idiotype v can 
survive, if it on the one hand receives at least a mini- 
mum amount of stimulation from the network, but is not 



C. Coins in a box 

Imagine a box and a heap of coins put inside initially. 
Then the box will be shaken, all overlapping coins be 
taken out and randomly generated new heaps of coins 
be placed on randomly selected empty areas of the box. 
This procedure will be iterated several times. Clearly, 
in the above algorithm, the box corresponds to the base 
graph G, coins correspond to occupied vertices on G, and 
empty areas to holes. Overlapping coins have too many 
neighbours, thus too high a degree and will consequently 
be taken out. Since also isolated coins are allowed to re- 
main, the situations is that of ti = 0. represents the 
maximum amount of coins which can be adjacent on a 
plane without overlap. 
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III. ANALYZING THE DYNAMICS 

In this section we focus on analyzing the population 
dynamics and understanding corresponding changes in 
the network structure. Figure |l|a shows a sample trajec- 
tory for the initial 300 timesteps of the population dy- 
namics for {ti,tu) — {l,7) and / = 10 on a base graph 
G^i2 ■ there seems to be a generic evolution with 
time it makes sense to look at averaged trajectories, i.e. 
("t) — X]n"'Pt(") where ptin) denotes the probability 
that after t iterations n occupied vertices will be present 
(see Fig. 0b). 

Clearly, several phases of the dynamics can be dis- 
tinguished. First, it takes a certain time till acciden- 
tally at least a connected pair of occupied vertices is 
formed which then functions as a germ for further net- 
work growth. 
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FIG. 1. (a) A sample trajectory for the population nt, the 
giant cluster, the 2-tree-reduced component of the giant clus- 
ter ('backbone') and its 2-leaves, see text. Parameters are 
d = 12, (ti,iu)=(l, 7) and I = 10. (b) Averaged trajectories 
of the population, the size of the greatest cluster and the av- 
erage cluster size vs. time. Averages have been taken over 
1000 independent runs, parameters are d — 8, {ti,tu)~{l,5), 
1 = 6. 



To elucidate the structure of this germ for ii > 1 we 
define three subsets of a cluster C: (i) its (generalized) 
ti-leaves kiiC), (ii) its (generahzed) fi-stem Sti(C) and 
(iii) its (generalized) ii-tree-reduced component rtj(C). 



Leaves of the first generation are vertices of C with less 
than ti neig hbours, i.e. lll\c) = {v e C\dv < h}. 

(2) 

Then, leaves of the second generation l).^ ' (C) are vertices 
of C\;p'(C) with less than ti neighbours. Analogously 
for n = 2,3,..., l[f ^ {v e C\l[^-'^\d^^^^i^-,,v < U}, 
where denotes the number of occupied neigh- 

Thus, the tree-like component 



(1) 



hours of V in C\ll^ 

of C is tt,{C) = [jjl, lif\C), the stem st, = - 
and the tree-reduced component r^j = C\it|. For tj = 2 
these notions coincide with the usual definitions in graph 
theory. 

Consider the application of the window- algorithm with 
lower threshold ij. If not sustained by random influx, the 
ti-stem of a cluster is prone to successively fall victim to 
cascades in which the ii-leaves are taken out first. Ver- 
tices which are critical, i.e. are still allowed, but will be 
removed if they lose a neighbour represent the {t\ -1-1)- 
leaves. Contrariwise, the (ii -I- l)-tree- reduced component 
is the subset not endangered by avalanches caused by 
critical vertices. 

Apparently, the tree-reduced component of a cluster is 
invariant under step (ii) of the window-algorithm. So, for 
ti > 1 in the above context the germ which for tj = 1 is 
just a connected pair of occupied vertices corresponds to 
a cluster with a nontrivial tj-tree- reduced component. In 
case of, e.g., ti — 2 this is a 4-loop, for = 3 a cube etc. 
The higher ti the more 'organized' a germ is demanded 
and the longer it takes till it is formed perchance. 

Initially, as this germ is still small it grows only slowly, 
since some of the new vertices are short of neighbours 
and thus are immediately taken out again. Then as a 
second phase of the dynamics a phase of almost linear 
growth of the population is entered. During this period 
almost all freshly thrown in vertices survive, since the 
network is already almost dense (thus providing enough 
neighbours) but still most of the vertices have only small 
degree. As the number of vertices with higher degrees 
increases the growth of the population then abates and 
reaches a maximum. It can also be seen from Fig. ^ that 
during this phase almost all occupied vertices belong to 
one giant cluster. At the same time {t « 150) as the 
maximum is reached, the giant cluster decays and small 
components start to split off. Interestingly, whereas in 
the initial stages of the dynamics the number of critical 
vertices is always of the same size as the influx, preceding 
the breakdown of the giant component more and more 
critical vertices are assembled (cf. Fig. |l|a). The maxi- 
mum number of critical vertices is assumed at the same 
time {t Ri 290) as the giant cluster breaks down. 

The third phase now ensuing is marked by the com- 
plete decay of the giant cluster and a coinciding drop of 
the population. As soon as this is achieved the smaller 
fragments start becoming rearranged, again associated 
with an increase in the population. 

Finally, the system saturates into a stationary state 
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around which fluctuations occur. The higher the more 
the population is allowed to grow initially and the more 
expressed the above described behaviour becomes. Inde- 
pendent of the upper threshold always half of the vertices 
are occupied in the final state. Recurring to clustering 
properties it turns out that average as well as maximum 
cluster size tend to 2.0. Thus, since there can't be a clus- 
ter consisting of more than two occupied vertices this 
proves that for long times Ft completely decays into 2- 
clusters. 

Holes play an important role in the systems approach- 
ing the steady state. A hole h will be called stable if it 
has degree dh > tu- Clearly, unless it loses its property 
to be stable, a stable hole cannot become occupied. Re- 
verting a stable hole back into a 'normal' hole demands 
the removal of dh — of its occupied neighbours, which 
itself needs a rearrangement in the 1-neighbourhood of 
these vertices. Thus, two characteristics of stable holes 
become obvious: (i) a hole is the more stable the higher 
its degree and (ii) whereas the change of a normal hole 
into an occupied vertex demands only a rearrangement at 
exactly the holes place, the occupation of a stable hole 
requires a rearrangement of its whole 2-neighbourhood 
and needs at least two iterations to be completed. 
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FIG. 2. Averaged trajectory (over 1000 independent runs) 
of the number of stable holes and of the population for d = 8, 
{ti,tu)—{l, 5), 1 = 2. The dotted line indicates half the system 
size IG^^Vs = 128. 



Figure ^ shows an averaged trajectory for the evolution 
of the number of stable holes and the population size. 
Surprisingly, even during the phase of declining popula- 
tion (which means that fewer occupied vertices to make 
a hole stable are available) the number of stable holes 
increases till finally in the steady state all holes tend to 
be stable. This can be understood from the above con- 
siderations. As described above in detail, the reversion 
of a stable hole into a normal hole requires occupied ver- 
tices in the neighbourhood of the hole to be removed. 
However, the removal of vertices becomes more difficult 
the more stable holes are in their neighbourhood. Thus, 
apparently, stable holes situated in each others (up to 
2-) adjacency can promote mutual stability. As a con- 
sequence of this a formation of almost 'frozen' domains 



which are made of mutually stabilizing stable holes can 
be expected. These domains compete with each other 
and finally one of them prevails and fills the complete 
system. Indeed, the arrangement of occupied vertices 
and holes in the steady state exhibits patterns which will 
be discussed more closely in the next section. 



IV. THE STATIONARY STATE 

In order to characterize the network structure of the 
stationary state every vertex w G G is assigned a mean 
occupancy rate s{v) = l/{Ti — To) ^^j,^ st(u) and a 

switch rate r{v) = l/(Ti - To)X;f=To(l " Sst{v),st+^{v))- 
Mean occupancy rate and switch rate give a measure of 
how many times a vertex is occupied or changes its occu- 
pation during the time interval [To, Ti]. Tq will be chosen 
such that the system has already reached the stationary 
state. This method provides a way of obtaining an aver- 
age picture of the occupation of G. 
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FIG. 3. Mean occupancy rate and switch rate for every ver- 
tex after a simulation on Gg^' with parameters (fi, /:u) = (l, 4), 
1 = 5. The relaxation time was chosen To = 10*, then 
averages over Ti = 9 x 10'* timesteps have been taken. 
Bit-strings v = {vi,...,vs) are uniquely represented by in- 
tegers z = ZiJ^o ^^'2'. 



Figure |^ shows the mean occupancy rate and switch 
rate of every vertex after a simulation on G^g\ Clearly, 
vertices with high mean occupancy rate have a low switch 
rate and vice versa. Thus, vertices which are frequently 
occupied tend to be occupied almost permanently; sel- 
dom occupied ones to be almost always holes. 
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To elucidate the structure of the network formed by 
the highly active vertices we define the subset S{a) = 
{v e G\s{v) < a} containing all vertices which are more 
frequently occupied than with rate a. 

In Fig. ^ the number of vertices belonging to S{a) and 
the greatest connected cluster of S{a) are shown as a 
function of the threshold value a. Due to the logarithmic 
scale a first drop from 15(0)1 = 256 to |<S'(a)| = 130 at 
a = is omitted. Two further thresholds become appar- 
ent: one at oi « 3 X 10"'' where the size of the greatest 
component of S{a) rapidly drops to size 2 and a second 
one at 02 w 0.893 which marks the decline in the dis- 
tinction of high- and low mean occupancy vertices seen 
in Fig. |. 




0.0001 0.001 0.01 0.1 
occupancy threshold 

FIG. 4. Number of vertices belonging to S{a) (see text) 
and the size of the greatest connected component of S{a) as a 
function of the threshold a. Data taken from simulations on 
Gg'^ with the window (ti, tu) = (l, 4) and influx 1 = 5. Due to 
the logarithmic scale the first point showing a drop of [^(a) 
from \G\ = 256 to 130 is left out. The dotted line indicates the 
level \S{a) \ = 128. Data have been assembled for Ti = 9x 10"' 
iterations after a relaxation time To = 10*. 




FIG. 5. Illustration of the cluster structure of the defect 
caused by the two vertices with mean occupancy rate s « si 
(drawn in grey) for n = 6. These vertices are in the centre of 
a star-hke cluster of other low-mean occupancy vertices from 
Li (empty circles). For a > ai they no longer belong to S{a) 
and thus their removal causes a decay of this cluster into ten 
2-clusters of holes. 



In order to get rid of fluctuations occuring during the 
iterations, one can distinguish a set of high mean oc- 
cupancy vertices Sh = S{si) (which can be considered 
as permanently occupied) and a set 5^ = G\Sh (corre- 
sponding to permanent holes) of vertices with low mean 
occupancy. From Fig. |^ one derives further that Sl 
decays also into two groups, namely, a set Sli of 128 
vertices with mean occupancy and two vertices (seen 
as a drop of |S'(a)| from 130 to 128 at a = ai) with 
somewhat larger mean occupancy a slightly smaller than 
ai. These 2 connected vertices having each k — 1 = 8 
neighbours from Sli are in the centre of a star-like clus- 
ter of low mean occupancy vertices (cf. Fig. They 
represent a vertex-pair normally belonging to Sh which 
have accidentally been taken out and not got refilled yet. 
Thus they form a defect in the expression of a pattern 
structure (see section V A ). 

A further clustering analysis of Sh and Sl reveals that 
both subsets of G consist only of 2-clusters. Therefore, 
every vertex of v Cz Sl has exactly dv — k — 1 neighbours 
from Sh and thus forms a stable hole of maximum pos- 
sible stability. This justifies the above notion of a frozen 
domain. 

In case of iu = k — 1 a holes being stable would require 
at least k = tu + I of its neighbours to be occupied. Due 
to the 2-cluster nature of the pattern, however, only k — 1 
neighbours are available, so that holes cannot be stable 
in this case. Patterns then become extremely transient 



in the sense discussed below in section V B 



A. Metastable patterns and pattern stability 

The actual steady state pattern, classified by group- 
ings of vertices with clearly distinguished mean occu- 
pancy rates, depends on the infiux /. If / is very small 
compared to the system size \G\ also patterns different 
from the 2-clustered pattern appear. The variety of such 
patterns is very abundant for ti = 0, becomes smaller 
the more restrictive the window is and increases with de- 
creasing ratio I/\G\. 

They prove to be metastable, i.e. relax to a more sta- 
ble pattern and finally to the 2-cluster pattern on very 
long timescales. Interestingly, for tj = both sets Sh 
and Sl prove to have exactly alike properties and thus, 
the system is symmetric against an exchange of occupied 
vertices and holes 

Figure |^ gives examples for such a metastable pattern 
together with a visualization of the ismomorphic graph 
structures of Sh and Sl- Typically, they consist of a 
large number of small clusters and few larger compo- 
nents. In the organization of the network structure of 
subclusters of Sh and Sl basic units (e.g. 6-loops in 
Fig. ^ play a role. Nevertheless the majority of the 
clusters turns out to be asymmetric. 

In this context a comparison to random graphs on G il- 
lustrates the very high organization of these patterns. A 
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random graph T C G (constructed by occupying vertices 
of G with a uniform occupation probabihty p) will con- 
tain almost always only one giant connected component 
for p > pc '-^ 1/k P^ . Here, multiple giant components 
exist, although the fill ratio |i7|/|G| = 0.5 far exceeds the 
percolation threshold. 

Stability of a pattern is determined by the stability of 
its stable holes. Thus, a (rough) measure for the stabil- 
ity of a pattern is, e.g., the average degree of its holes 
{dv)y^SL ■ It turns out that all these metastable patterns 
have an average hole degree slightly smaller than that of 
the 2-clustered configurations. 




FIG. 6. Nontrivial clusters wliicli appear in metastable pat- 
terns for 1 — 5, (ti, tu)=(0, 7) on G^^ . (a) 5- and 16-clusters 
from a pattern built by 192 singletons, 64 5- and 32 16-clus- 
ters. (b) 14- and 44-clusters from a pattern with with 208 1-, 
48 5-, 16 14- and 8 44-clusters. (c) 2- Tree-reduced component 
of the 203-cluster from a pattern consisting of 152 1-, 12 2-, 
12 3- and 4 203-clusters (2- and 3-clusters form chains). In 
the cases (a) and (b) larger clusters contain structures which 
resemble the smaller clusters. Larger loops, especially 6-loops 
and trees play a distinguished role. In all cases the dimension 
of the cluster structures is remarkably low. 



Figure [7| shows another way to quantify stability. For 
this purpose we apply a test influx /. More specifically, 
a pattern P of holes and occupied vertices is taken, a 
number / < \L\ of its holes h belonging to the low 
mean occupancy set Sl randomly selected and occu- 
pied, the window algorithm applied and the distance 
GG ~ sp'{v)\ between the original 

pattern and the resultant graph P' measured. Intuitively, 
/ is a measure for a perturbation which is applied to the 
pattern. A small mean deviation {d{P, P')) means a high 
stability against random influx /. For growing system 
sizes the sigmoid function {d{P, P')) /\L\ tends to a step 
function. 




0.2 0.4 0.6 0.8 1 
relative influx 



FIG. 7. Relative mean distance {d{P, P')) /\L\ as a func- 
tion of the relative influx I /\L\ for patterns which arise on 
G[^\ Different lines correspond to patterns with different 
mean cluster sizes (c) which emerge for the given combina- 
tions of thresholds {ti,t^), see text. For given relative influx 
patterns are the more stable the smaller their relative mean 
distance. 



Confirming the above ideas. Fig. [7| visualizes two ten- 
dencies: (i) an upper threshold iu < k — 1 closer to k — 1 
Q and (ii) a higher mean degree of holes {dh)heSL pro- 
mote the stability of patterns. Indeed, patterns with the 
same mean hole degree have almost the same stability 
characteristics {d{P, P')) /\L\. For example, the patterns 
with {ti,U) = (0,7), (c) = 8/3 and (1,7), (c) = 4 whose 
curves coincide in Fig. |^ have both a mean hole degree 
{dh)heSH = 8.5. 

Due to the high supply of new idiotypes from the bone 
marrow these patterns are without interest for the pur- 
pose of modelling immune networks. However, it could 
be conceived that they prove interesting in the context 
of evolving species interaction networks where the influx 
as the mean number of new species per time unit is small 
in comparison to the overall network size. 

B. Stable base configurations 

The 2-clustered patterns are distinguished in two re- 
spects: (i) they are the only patterns evolving if the in- 
flux is high and (ii) they are stable, i.e., if once such a 
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pattern is assumed the system will not relapse to one of 
the metastable patterns. Therefore we call it stable base 
configurations. 

The complete decay of both hole- and vertex- 
configurations into 2-clusters allows it to calculate the 
multiplicity of these base configurations. It turns out 
that the global structure of a base configuration is al- 
ready determined locally by fixing the occupancy sq of 
one connected pair of vertices. By the knowledge of the 
2-cluster structure of Sh and Sl one then knows that the 
vertices in the neighbourhood of this pair have occupancy 
Sl = 1 — Sq. Thus, also the occupancy of their 2-cluster 
mates is fixed, which in turn determine the occupancy of 
vertices in their neighbourhood and so on. By this ar- 
gument every vertex' occupancy is uniquely determined 
since G is connected. All base configurations constructed 
in this way have been found to occur and are isomorphic 
to each other in the sense, that without labelling vertices 
they can't be distinguished. 

As a consequence of the above argument, fixing sq and 
chosing one arbitrary vertex o e G and a link associated 
with a bit-operation P leading to a two-cluster mate of 
o, P{o), base configurations can be denoted by B(^gg py 
Hence, the number of possible base configurations is 2k. 
The system is degenerate, though as 2k/\G\ for 
|G| ^ oo not macroscopically. 

From a biological point of view it appears appeal- 
ing to identify 2-clusters with idiotypc-antiidiotype pairs 
the antiidiotype playing the role of an internal image 
(memory) of a previously encountered antigen ||2l|,^,^ . 
The 2-clusters found here, however, are 'coherent' in a 
fixed pattern of a stabe base configuration. This implies 
a maximum storage capacity 2 k which -also for large 
systems- is far too few to account for experimental ob- 
servations. 

It appears useful to consider distances between base 
configurations. One finds: 

r|G|/2 ifPy^QorsoT^si 
d{B{so,P),B(^si,Q)) ^ \\G\ if P = Q and sq = 1 ~ si 
[O if P = Q and so = si- 

(3) 

Therefore, except the 'inverse configuration', base con- 
figurations differ in exactly the half of their sites from all 
other base configurations. For the case P Q or Sq si 
half of the differing vertices are holes and half occupied 
vertices. Thus, the difference between base configuration 
is of the order of the system size and a change from one 
base configuration to another cannot take place without 
a major reorganization. 

As the influx / increases occupied vertices become 
more and more prone to be removed. So, it can be 
expected that the distinction between highly active and 
scarcely active vertices loses sharpness. Increasing / over 
a threshold leads to a completely different behaviour of 
the system. Parameter regimes in which the system ex- 



hibits qualitatively different dynamics will be discussed 
in the next section. 

V. DEPENDENCE ON THE INFLUX / 

The systems behaviour depends crucially on the clus- 
tering properties of Fj. To obtain a picture of the dy- 
namics of cluster changes the time-sampled cluster size 
distribution over the interval [Ti,To], p{c) — l/(Ti — 
^o) Z^t^^To Z^Ke '^cPt('^c, c) where pt{nc,c) denotes the 
probability that Ft contains ric clusters of size c, has been 
investigated. Figure || shows numerical data for the clus- 
ter size distribution obtained for five different values of /. 
For small infiux / = 20 (Fig. |8^) the systems behaviour 
is dominated by the occurence of 2-clusters. Occasion- 
ally also other clusters of sizes c = 6, ...,11 and sizes 
c = 16, 20 appear. For / = 30 (Fig. |b) the distribu- 
tion has a structured tail of series of local extrema. Then, 
for / = 40 (Fig. ||c) the 2-cluster dominated structure is 
still expressed, but now also clusters some orders of mag- 
nitude larger are found. Finally, for / = 80 and / = 120 
(Fig. ||d), the clustering structure is dominated by (de- 
pending on ti and t^) one or several giant clusters and 1- 
clusters (singletons) which more and more outweigh other 
small clusters as / is further increased. / = 20, 40 and 80 
represent three different parameter regimes for the influx 
which will be treated below. 




cluster size cluster size 

FIG. 8. Numerical data for the cluster size distribution for 
different influxes / = 20 (a), 30 (b), 40 (c), 80, and 120 (d) 
corresponding to different regimes as described in the text. 
Simulations were performed on Gg^^ with (ti, tu) = (l, 5). 
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A. Small /: A statistical approach of defects 



1. The case ti — O 



For small / during the initial stages of the dynamics the 
2-cluster dominated pattern as described in section IV B 
is formed. Once such a pattern is established / works as 
a perturbation which incidentally causes defects to the 
ideal pattern structure. These defects can be classified 
into two groups: 'hole'-defects and 'vertex'-defects (cf. 
Fig. ^. Clearly, hole-defects are caused by k — 1 — tu 
correlated vertex-defects and thus are much less proba- 
ble than vertex defects. As long as only vertex defects 
occur only 1- and 2-clusters can be formed. However, 
once a hole defect is established, it serves as a junction 
for its surrounding 2-clusters and clusters of size 2i/ -I- 1, 
v — 1,2,..., K are built. For example, in Fig. local 
maxima are found for c = 7,9, and 11. The latter one 
corresponds to the most probable kind of hole defect: a 
formerly stable hole now is occupied and has five remain- 
ing occupied neighbours. In the case of the less probable 
cluster of size c = 9 four once occupied neighbours of 
the hole are defective, thus four are remaining. Analo- 
gously for c = 7. As the 2-clusters surrounding a hole 
defect might also be defective though less frequently also 
clusters of even size appear. The second series of local 
maxima around c = 20 in Fig. |^a can be explained by 
hole defects which are correlated, i.e., hole defects whose 
occupied 2-cluster neighbours are connected. 

This microscopic picture of the process suggests a 
treatment borrowed from equilibrium statistical mechan- 
ics. Since defects are more likely to occur for higher in- 
fiux, / can be associated with a kind of a 'temperature' 
T{I). T{I) will be growing monotonically with /. Defects 
are assigned an 'energy' a(tu) which describes the prob- 
ability that a defect is caused. Since it is the harder to 
remove occupied vertices the higher it is also expected 
that a grows with increasing upper thresholds tu- 

In the following we derive a statistical description for 
an ideal gas of defects. 




FIG. 9. Illustration of 'hole-defects' (left) and 'ver- 
tex-defects' (right) for (ti, tu)=(0, 4). The defective vertex 
is printed in grey, holes in white and occupied vertices in 
black. At the left hand side one of the central holes has five 
occupied neighbours and thus is stable. The grey hole has 
lost two of its neighbours due to 'vertex defects'. Only three 
neighbours remain so that the grey hole has lost its stability 
and can be occupied. Both unstable and occupied holes are 
called hole-defects. At the right hand side the grey vertex got 
removed, causing thus a defect to the ideal 2-cluster pattern 
structure. 



The probability to cause a vertex defect depends on 
the number of holes surrounding the occupied vertex. In 
spite of this no distinction between defects of completely 
removed 2-clusters and singletons will be made. This is 
justified since for large systems an occupied vertex which 
remains after its 2-cluster mate has been removed is still 
surrounded by almost the same number of holes. Fur- 
thermore, for t\ — Q the algorithm allows the permanent 
existence of singletons. 

The probability of having I independent defects of en- 
ergy a at temperature 1/(3{I) in an ideal pattern of 
N = \G\/2 occupied vertices is 



1 fN 



where the partition sum Z is easily determined as 



N 



/=0 



= (1 



-,9a\JV 



Then the mean number of defects is given by 



{1} 



1 dlnZ 



(3 da 



N - (n) = N 



-Pa 



1 



-13a ■ 



(4) 



(5) 



(6) 



{n) being the mean number of actually occupied vertices. 

Comparing with data obtained from simulations it 
turns out that the ansatz 1//3 = /g -|- J holds for a broad 
range of different values of /. From a fit of the mean 
populations with Eq. (^ one obtains a — 88.14 ± 0.48 
and Iq = 0.67 ± 0.18. In the whole range up to / < 35 
mean values and probability distributions calculated by 
Eq. (^ are in good agreement with simulations (see Figs. 
|l|and|n]). 

For very small / it is by simple combinatorics possible 
to obtain an approximation for the 'energies' fitted from 
the data of Fig. 10, If for fixed the influx / is so small 
that it can cause not more than exactly one vertex defect, 
i.e. / < 2(tu -I- 1), the probability per site for a defect is 
ill 



Vd{I) 



N 



■eg 



K- l\ /iV- (k- 1) 

3 ) \ I-J 



(7) 



If no more than one defect can arise, the mean number 
of defects is given by = Npd{I). Using Eq. (|^) one 

calculates Pd{t\i) and Pd(iu + 1)- A comparison with Eq. 
(P) yields 



(lnp<j(iu))"^ - (lnj5d(iu + 1))"\ 



(8) 



which, e.g., evaluated for d — S and tu = 5 gives a = 
50.8 which is of the same order as the numerical value 
a = 88.14. Comparing /q is not reasonable, since Eq. 
(|6|) bears deficits for small influxes. 
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10. Mean population (n) vs. 
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40 



FIG. 10. Mean population (n) vs. influx / taken from sim- 
ulations for {ti,tu)={0, 5) on Gg^'. The crosses represent sim- 
ulation data, the line corresponds to a flt of (n) calculated 
from Eq. (p|). Fit parameters are a = 88.14 and lo = 0.67. 
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FIG. 11. 

tions p(n) =l/(ri ---uj^t^rp^, 

size obtained from simulations on Gg^' for (ti, tu) = (0, 5) and 
/ = 30 (a) and / = 20 (b). The simulation results (crosses) 
are compared with p{N — n) from Eq. (lines connect cal- 
culated points) using a — 88.14 and Jo ~ 0.67. 



Indeed, the range of validity of Eq. (|6|) is limited in two 
respects. First, although the possible number of defects 
for small / is limited by [I /tu\ the sum in Eq. (^) extends 
up to N. However, for low temperatures surplus defects 
are counted with only a small statistical weight. This ap- 
proximation becomes worse for high upper thresholds t^- 
Secondly, too many vertex defects (which then become 
correlated) lead to hole defects. Hole defects destabilize 
the 2-cluster pattern and lead to a qualitative change of 
the above picture. 



S. The case ti = l 

For ii = 1 a distinction between two types of vertex 
defects becomes necessary. Now, unless a new 2-cluster 
mate is provided by fresh influx during the next iteration, 
a singleton violates the lower threshold rule. Hence it can 
only survive for one timestep. 

To cope with this situation in the present framework 
defects consisting of a singleton (a-defects) are assigned 
an energy a, defects made of a completely removed 2- 
cluster pair (7-defects) an energy 7. Both energies are 
expected to increase with growing bit-chain length d and 
for higher upper thresholds t^. Then a configurations 
energy with li a- and I2 7-defects is E{li, I2) = lia + l2j- 

The probability of having li independent a- and I2 7- 
defects is 

For the partition sum one obtains 

E 



N/2 , , , NI2-I1 , , 

fN/2\ ..p^ U N/2 - h\ 



^=i^\i, p'^^ ^ y 12 



= (1 + 2e-'3" e-''^)^/2 



(10) 



In analogy with Eq. (|^) one derives the mean number of 
a-defects 



(^i) = N- 



-0a 



1 + 2e"^" + e-z^T 
and obtains the mean number of 7-defects 



(^2) = 



N 



-M 



2 1 + €-0-1 + 26-/3" ■ 



(11) 



(12) 



Finally, the number of empty vertices is n = — — 
2^2, and hence the mean population is given by 



(n) = 



l + e" 



-(3a 



1 + e-f^-y + 26-^5" ■ 



(13) 



The ansatz = /q -I- / is again confirmed by a com- 
parison of Eq. ( |l3|) with numerical data for the mean 
population (see Fig. ^2|). The probability distribution 
that n vertices are occupied at a time t is 



prob{n occupied vertices} 



(2=0 



p{N -n -212,12). 



(14) 



where ^max = — n/2\ and p{li,l2) as given by Eq. 
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FIG. 12. The mean population for G[]^ with {ti,tu) — {l, 5). 
Fitting with Eq. leads to q = 632 ±99, 7 = 250.6 ±13.1, 
and lo = 36.1±4.2. The fit breaks down if one tries to include 
the range I > 150 w hich marks the onset of the transition 
region (see Sec. VB). 



Analytical results derived from Eq. (|lj) are again 
in good agreement with simulation data (see Fig. 13). 
Equations ( p^ ) and (|l^) are subject to the same limita- 
tions as in the case for ti — 0. 
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the formation of great clusters consisting of connected 2- 
clusters. As / becomes still larger, the probability that 
hole defects become correlated increases. Occasionally 
this leads to the emergence of a giant cluster (see Fig. ||, 
the different series of local maxima correspond to an in- 
creasing number of connected hole defects). Speaking in 
the language of thermodynamics, growing influx entails 
a growth of fluctuations which finally tend to overthrow 
the regular pattern structure. 
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FIG. 14. Simulation data for Gg^\ (ti,tu)=(l, 4). (a) shows 
the population n, the number h of stable holes, and the size 
of the greatest connected component of Ft, gc. Initially, ap- 
proximately half of all sites on become stable holes and a 
pattern is assumed, (b) shows a transition region. The num- 
ber of stable holes h and the population n drop suddenly, a 
giant cluster is formed, (c) shows the smallest and the second 
smallest distance to a stable base configuration mi and m2 
(cf. text). In the transition region no pattern is selected, (d) 
displays a number for the pattern which has the smallest dis- 
tance to Ff. In the transition region between two sharply as- 
sumed patterns the state changes quickly. For t £ [200, 1700], 

fcmin = 5, for t > 2000, fcmin = 4. 



FIG. 13. Comparison between simulation data for the 
probability distributions of the population size for G^o', 
(ti, tu) = (l, 5) (a) 7 = 25 and (b) 7 = 80 and results derived 
from Eq. ( p^ ) for the same a, 7, and lo as in Fig. |l2| In (b) 
the influx is already high enough that both types of defects 
appear with almost equal probabilities. 



B. The transition region 

As previously discussed increasing / beyond a thresh- 
old leads to a growing number of hole defects. The 
appearance of correlated hole defects is connected with 
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FIG. 15. Comparison between the mean occupancy distri- 
butions measured in a transition region (left) and during the 
time to the transition region (right). In the right hand figure 
a 2-clustered pattern is assumed. 
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The building of a giant cluster requires the reversion 
of a major fraction of the stable holes into normal holes. 
Thus, since stable holes form the 'skeleton framing' of 
the pattern structure, the system relapses into a phase 
where no pattern lasts for a longer timespan. 

Figure |l^ illustrates this behaviour for a simulation on 
Gg^'' with (ti, iu)=(l, 4). Initially, after the first stages of 
the dynamics the system settles into a steady state. For 
better illustration every stable base configuration B(s^ p) 
is given a unique number k,k — 1, 18. In Fig. [L4| c the 
minimum distance 

mi = mind(rt,Bfe) (15) 

k 

and the second smallest distance 

7712= min d(rt,Bk) (16) 

to any stable base configuration are shown. For the time 
intervals [200,1700] and [2000,3000] it holds mi < ma. 
Consequently the system is in the stable base configura- 
tion iJfe„i„ . Incidentally, hole defects associated with the 
emergence of larger clusters appear. Mostly, as long as 
hole defects remain uncorrelated, they are repaired dur- 
ing the next iterations. At t « 1700 the exact lock in of 
state fcmin = 5 is lost and for t G /change — [1700, 2000] 
a phase marked by frequent changes of fcmin is entered. 
During this period mi « m2 and thus no base configu- 
ration is sharply assumed. A comparison of vertex mean 
occupancy rates during this phase and the previous one 
justifies to speak of the time-interval t G /change as of a 
disordered period, since, clearly, any distinction between 
high mean occupancy and low mean occupancy vertices 
is lost. As expected, the emergence of the disordered 
phase goes hand in hand with the formation of a giant 
connected component in Ff , a drastic loss of stable holes, 
and a drop in the population. 

The whole dynamics of the system is marked by a se- 
quence of ordered and disordered phases. As / is in- 
creased, the residence time in ordered phases declines till 
finally the frequency of 'base configuration' changes (de- 
fined as a change of fcmin) tends to one. Thus, the system 
becomes completely disordered. 

Figure |l^ displays simulation data on G^g'' with 
(ii) iu)=(l: 8) for the mean time (residence time) during 
which fcmin remains unchanged. An abrupt transition 
from a phase marked by permanent patterns to a phase 
of transient patterns interrupted by periods of disorder 
occurs at Ic w 270. Accordingly, as can be seen from Fig. 
p^ , the mean lifetime (averaged over all vertices) drops 
abruptly during this range of influxes. For / > 300 mean 
lifetimes decay exponentially and quickly tend to one. 
Very short lifetimes, however, do not allow for static or- 
der or memory. Apparently, the transition from order to 
disorder goes through a phase of more and more frequent 
pattern changes. 
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FIG. 16. Mean residence time, i.e. time during which fcmin 
remains unchanged (excluding a relaxation time of the first 
5000 timesteps). All simulations on G^^ with (fi, tu) = (l, 8) 
have been allowed to run for 10^ timesteps. For I < Ic = 270 
no change of fcmin has been found. 
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FIG. 17. Mean lifetime vs. influx / obtained from sim- 
ulations on 

G^o' for (fi,tn)=(l,8). The mean lifetime was 
determined after allowing 5000 timesteps for relaxation to a 
steady state. For every / the simulations then run for 95000 
more iterations. Data for / < 100 are not shown since mean 
lifetimes exceeded simulation times. 



C. High / > Jci Iterated maps 

For influxes I > Ic the lifetime of occupied vertices be- 
comes very short. This indicates that the system can no 
longer form ordered patterns and long-time correlations 
between the systems configurations disappear. Thus, 
while for low influx clearly (nt+i) is an intricate function 
of both the systems configuration Ft at timestep t and 
the most recent infiux, for high infiuxes the mean popu- 
lation at timestep t -\- 1 tends to depend only on the size 
of / and the mean population at the previous timestep. 
Also, from the mean occupancy distributions one conjec- 
tures, that the structure of Ff is essentially random, i.e. 
obtained by randomly occupying [Ff | vertices of the base 
graph G. 

If one randomly occupies n sites of a base graph G 
the relative mean number h{n) of occupied sites after 
sites with degrees outside of (t\, t^) have been removed is 
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given by 




if n < \G\ 

otherwise. 
(17) 

Considering a large random influx, apart from fluctua- 
tions the relative population xt will evolve according to 
Xt+i = h{I/\G\ + Xt). Thus, in the steady state x will 
preferably assume values near the stable fixed points of 
the shifted function g{x) = h{I/\G\ + x), i.e. near the 
solutions of X* = g{x*). Linear stability analysis yields 
that a fixed point is stable if < 1. Analogously, n- 

cycles {x\gix*),g('Hx*) = g{gix*)), g^^~^Hx*)} are 
solutions of (7^"-'(a;) = x and are stable if Ig'^^^'ix*)] < 1. 

For K = 11 and (ii,iu) = (lj8) 17(0;) is a function with 
one maximum. Consequently, one expects that g^"-* has 
at most n maxima. Thus, in principle, cycles of higher 
order are possible. 
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FIG. 18. The bifuraction diagram obtained by iterating the 
function g{x) in comparison to simulation results for the max- 
ima of the population histogram for (ii,tu) = (1,8) on G^o' 
(filled squares connected by lines). Solid lines correspond to 
stable branches, dashed lines to unstable branches. The re- 
gion of the simulation data where three stable solutions co- 
exist corresponds to the onset of disorder: The intermedi- 
ate branch represents the mean population during periods in 
which the system closely assumes a base configuration, the 
upper and the lower branch are data sampled during disor- 
dered phases. Especially for higher influx simulation results 
and the data obtained by iterating g are in very good agree- 
ment. The smaller diagrams visualize changes in the stability 
of fixed points of g at the branching points: (a) g (solid line) 
becomes unstable at I/\G\ — 0.2, but g^^'' (dashed line) is 
stable. Hence a cycle appears, (b) At I /\G\ = 0.3 also (7'^-* 
(solid line) becomes unstable and -since gr'^-* (dashed line) 
gives only one unstable solution- a 4-cycle emerges, (c) At 
I /\G\ = 0.45 g^^^ becomes stable again leading to a backward 
bifurcation and a 2-cycle. 



gf^"^ is of order and there is no general analytic solu- 
tion to X* = (;^"^(x*). As an example we consider the 
case (ti,tu) = (1,8) on g[1\ i.e. k = 11 and |G| = 1024. 
Evaluating the fixed point equations numerically, one ob- 
tains the bifurcation diagram displayed in Fig. At 
X — 0.2 one finds a forward pitchfork bifurcation towards 
a 2-cycle, which then at a: = 0.3 becomes also unsta- 
ble, leading to the emergence of a 4-cycle. For x = 0.45 
this 4-cycle is found to become unstable and a backward 
pitchfork bifurcations leads again to a stable 2-cycle. Far 
outside the region of biological interest, for x = 0.93 a 
backward bifurcation occurs and the single fixed point 
solution again becomes stable. Figure |l^ also displays 
data obtained from simulations of the window algorithm 
on G^Q^ which exhibit the same qualitative behaviour as 
the iteration of g. As expected, deviations become small 
as / increases. 



VI. MORE DENSELY WIRED SYSTEMS: 
TWO-MISMATCH BASE GRAPHS 

In this Section some observations about the dynamics 
created by the window algorithm with ii — 1 on more 
densely wired base graphs will be presented. As a conse- 
quence of a higher coordination number different static 
patterns and a new type of dynamic pattern can be ob- 
served. In principle, we expect (i) more sparsely popu- 
lated, and hence transient patterns of reduced stability 
for low tu and (ii) enhanced stability for upper thresholds 
close to K - 1 (cf. Sec. |IVA[ ). 

As an extension of the base graphs G^^"* whose link 
structure is created by 'one-mismatch links' we now in- 
troduce additional links connecting bit-chains which de- 
viate in two positions from complementarity, so defining 



(2) 

the base graphs G)j ' , cf . Eq. 

.(1) 



Clearly, since G)^ is connected such 'two-mismatch 

links' of G^^'' are shortcuts of paths in G^'' . For example, 
referring to the notion of bit-operations associated with 
links (see Sec. II A), two vertices linked by Lfeifca G G^ , 



ki 7^ fe, are also linked by the six paths corresponding 
to combinations of the edges Lq, L^^, Lk^ G G^K 

Although every vertex now acquires d(d — l)/2 new 
edges the former link structure is conserved. Thus, it 
could be conceived that patterns created on G^ re- 
tain a part of the structure of the stable base configu- 
rations which are formed on G^J^\ Counting, e.g., 'two- 
mismatch' links in a base configuration B(^sg p^j C G^J''^ 
one finds that every occupied vertex now acquires d 
new links on G^^^ (cf. Fig. |l^), aggregating to a to- 
tal degree d + 1. Thus, this kind of patterns is forbid- 
den for tu < d + 1. However, it turns out that apart 
from metastable structures for small infiux the dynam- 
ics always ends up in the above described configurations. 

(2) 



rjii r ; • ^ \ ■ 1 • 1 r J J- rru Tfius, foY > d + 1 thc base-graph G\ always decays 

ihe function g[x) is a polynomial of order t^. Ihus j u _ & r- </ j j 
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into two groups of high mean occupancy vertices Sh and 
low mean occupancy vertices Sl- Both, Sh and Sl form 

(2) 

{d+ l)-regular s ubclu sters of G)j . Resuming the consid- 
erations of Sec. [VB one finds that these stable base con- 
figurations on G]^' have multiplicity 2d. The scenario for 
growing influx is the same as on G^^-* : for I > Ic phases 
of well-defined base configuration are interrupted by dis- 
ordered periods leading to pattern-changes, and finally to 
a dynamics marked by randomness and very short life- 
times. 



O oLij 



O- 



6 6 




FIG. 19. An illustration of how 'two-mismatch links' 
(dashed line) contribute to the degree of occupied vertices in 
a base configuration _B(sg_ij). Occupied vertices in B^so,Li) 
are drawn in black, empty ones in white. Two-clusters (e.g. o 
and p) in B are connected via the 'one-mismatch link' (solid 
line) Li. Except o the one mismatch neighbours hj = Lj{p), 
j 7^ 1, of p are holes. Apart from the holes' 2-cluster mates 
Li{hj) all their neighbours are occupied. If one pursues an 
inversion link Lo from any such hole hj adjacent to p, a vertex 
q = Fo(hj) is reached which is -via Lij- also a two mismatch 
neighbour of o. One can easily verify that o has no other oc- 
cupied 'two mismatch neighbours'. Consequently, in a base 



configuration B on 
cupied neighbours. 



(2) 



every occupied vertex has d + 1 oc- 



However, for < d+1 where 'relics' of patterns form- 
ing on gII"^ are no longer allowed the behaviour turns out 
to be more interesting. 
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I 3-5-group patterns 
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FIG. 20. Mean lifetime (log-scale) vs. infiux for simula- 
tions on with {ti,tu)~{l,W)- Successively, the system 
forms cube-patterns, multiple-group patterns, singleton pat- 
terns, and again multiple-group patterns. Then, patterns be- 
come transient and finally a randomness-driven regime is en- 
tered (see text). 



As tools to investigate the structure of networks we 



again consider the mean occupancy rates of all vertices 
and probe the mean occupancy-patterns by defining sub- 
sets S{a) C G_y with different threshold-occupancy rates 



a (see Sec. FV). In analogy with results for the one- 
mismatch case we find metastable patterns for small in- 
fluxes. Depending on the ratio I/\G)i |, first completely 
2-clustered, 4-clustered or 8-clustered patterns appear (4- 
clusters are chains, 8-clusters cubes). Next in a range of 
influxes cube-configurations dominate (cf. Fig. |2^). Dif- 
ferent from the 2- and 4-cluster patterns, however, the 
cubes don't fill the base-graph completely and 256 ver- 
tices not matching in the cube pattern are left. 
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FIG. 21. Every vertex' mean occupancy obtained by a sim- 
ulations on 

G^2'' with (ti,tu)=(l, 10) and I = 30. Vertices 
are labelled by integers z corresponding to their respective 
bit-chains. The relaxation time was To — 5000, mean oc- 
cupancies have been sampled over Ti — To = 15000 further 
timesteps. An offset has been added to display the group Sl 
with the lowest mean occupancy (a)sj^ — 0. The intermediate 
group Sm has mean occupancy (a)sM ~ 0.01, the group with 
highest mean occupancy (a)si, ~ 1.0. 



Figure 21 shows data for every vertex' mean occupancy 

(2) 

obtained by a simulation on G\2 with {ti,tu) = (1, 10). 
The set of vertices with highest mean occupancy Sh de- 
cays into cubes, the set with lowest mean occupancy 
Sl forms a giant component of 2816 vertices. Sl con- 
sists of 1536 almost never occupied vertices of degree 



\G\Sl 



= 61, 256 with d\ 



d\G\SL = 39 



G\Sl 



= 43 and 1024 with 



All degrees are larger than and thus 
Sl entirely consists of stable holes. The 256 vertices not 
exactly matching in the cube pattern form a subset Sm of 
intermediate mean occupancy. Sm turns out to contain 
only singletons, each of which is surrounded by elements 
of Sl which are almost never occupied. Thus, vertices 
of Sm are isolated spots of activity sustained by infiux 
temporarily placed at their surrounding stable holes, but 
taken out immediately afterwards due to overstimulation. 
Consequently, as long as no fresh influx is placed adja- 
cent to it in the next timestep a vertex of Sm dies out 
one timestep after it has been inserted. This view is 
conflrmed by very short mean lifetimes {Tii{e)sM ~ 1 of 
elements of Smi which slightly grow with increasing in- 
fiux. Together with a growing probability that isolated 
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holes get filled this accounts for relatively large mean 
occupancies {a)sM in comparison to {oJsl- Though at 
the first glance surprising that a vertex can survive at a 
site where it is surrounded by almost always empty sites, 
this but indicates a mechanism which is to prevail in a 
consecutive regime: isolated vertices of high durability 
are sustained by quickly fluctuating short-living influx in 
their neighbourhoods. 




1000 2000 3000 4000 
vertex z 



FIG. 22. Mean occupancy rates of vertices z derived from 



a simulation on Gfa' witli = 10) and / = 100. Six 

(2) 

groups of vertices S\,...,Se C can be distinguished by 

their mean occupancy. 

Increasing / over approximately 80 the cube patterns 
become unstable and from / « 100 onwards a pattern 
marked by 6-levels of mean occupancy (see Fig. 22) is 
chosen. In contrast to the appearance of several levels of 
mean occupancy due to transitions between isomorphic 
stable base configurations this grouping is of a different 
nature. While in the first case simulations started with 
different initial conditions lead to different numbers of 
groups and changing mean occupancy levels (depending 
on the timestep when transitions occur) simulations per- 
formed with different initial conditions now always repro- 
duce these 6 groups and their mean occupancies. 



roup S 


|S| 


(C) 


{r) 


(r) 




(C*) small 


(n) 


Si 


1124 


371.0 


0.0 


1.0 


0.0 




0.05 


S2 


924 


371.0 


3.8 


0.26 


0.0 


7.0 


27.2 


S3 


924 


371.0 


5.4 


0.18 


0.0 


4.6 


61.3 


Sa 


134 


1.0 


10.0 


0.1 


1.0 


1.0 


26.5 


S5 


330 


160 


18.1 


0.06 


0.5 


1.007 


107.05 


56 


660 


260 


35.3 


0.028 


0.27 


1.03 


318.16 



TABLE I. Data characterizing the 6 groups 5*1,..., Se ob- 
tained from simulations on G'"^^ with (ti, tu)=(l, 10) and 
I = 100. In 30000 timesteps vertices of Si have always be- 
longed to the greatest subcluster of Ft, hence {C)sniaii could 
not be measured. Since they are immediately taken out after 
they got into the system stable holes have a mean lifetime 
(r) = 0.0. This means that there mean lifetime is shorter 
than the discretization of time. 



The interval of influxes / € [80, 100] is characterized by 
transitions between both types of patterns in the course 
of which for higher influxes periods during wich the 6- 
group pattern is assumed more and more prevail. The 
change of the mean population averaged in both types 
of pattern is drastic: for / = 90 the cube-pattern can 
sustain approximately (n) w 1024 vertices while at the 
onset of their appearance 6-groups patterns can support 
a mean population of only about {n) « 600 occupied 
sites. Comparing the mean occupancy levels of the 6- 
group pattern with all previous pattern structures one 
remarks that, while groups of always empty sites are 
still present, groups of almost permanently occupied ver- 
tices are lacking. All groups have mean occupancy rates 
smaller than 0.7. This, and the relatively short lifetimes 
of vertices belonging to the groups 84,85, and 5*6 with 
high mean occupancy, speaks in favour of an interpreta- 
tion as a 'dynamic' pattern. Occupied vertices belonging 
to different groups 'fluctuate' with different rates, while 
the interplay of them secures that the distinct role of the 
groups is preserved. In the following the nature of this 
dynamic pattern will be elucidated in more detail. 

Figure ^ and Table | give data characterizing this six 
groups: mean size of the cluster (C) on Ff that a ver- 
tex of a group belongs to, its mean lifetime (r), mean 
switch rate (r), the mean frequency (/small) of belonging 
not to the giant component and the mean cluster size 
(C) small of the small clusters if the vertex is not in the 
greatest component of Ft. On average elements of 81, 82, 
and 83 belong to 371-clusters on Ft, but are very seldom 
members of small clusters or singletons, have relatively 
short lifetimes and high switch rates. In contrast, ele- 
ments of the groups of higher mean occupancy 84,85, 
and 8e typically have smaller mean cluster-sizes, form 
more frequently small clusters, preferably singletons, and 
have long lifetimes. Mean lifetimes are increasing with 
increasing mean occupancy of the groups. 

Exploring the interconnectedness and network struc- 
ture of the groups 81,..., 8e one obtains the following 
picture (cf. Fig. [2^ ): The strongly interconnected groups 
5*2 and 5*3 form a core of G to which -except iS'4- all the 
other groups are attached. Excluding this core, each of 
the other groups consists of singletons. The largest set 
81 is a reservoire of stable holes and connects 84 with 
the core. Thus, vertices of 84 play a similar role as the 
previously mentioned isolated holes in the cube-pattern: 
they receive their whole sustenance from the influx and 
have no permanent connections. While the group of sec- 
ond largest mean occupancy, 85, has only links to the less 
populated part of the core, 82', the most active vertices, 
elements of 5*6, have connections with both parts of the 
core. 

Consequently, apart from stimulation by random in- 
flux elements of 85 obtain stimulus from the core-group 
83. Nevertheless, the stimulation is not optimal leading 
to a mean occupation of only 1/3 of 85. The group 8e 
receives stimulation from both core groups and so can 



15 



support a higher population than S5. Contrarily, since 
being strongly interconnected and connected to both the 
richely populated groups S^, and Sg most vertices of S2 
are suppressed. In the case of 5*3 which does not connect 
to S5 suppression is lower and thus a higher mean pop- 
ulation can be sustained. The data shown in Table || are 
in qualitative agreement with this interpretation. 



From / w 260 on these dynamic multiple group pat- 
terns become unstable and rearrangements between dif- 
ferent (isomorphic) configurations set in. Analogous to 
the scenario discussed in Sec. VB disordered periods 



Figure 24 displays simulation data for the change of the 
relative occupations Si = ^/\SiU2ves- ^(^)' * ~ li-'-jS, 
of the 6 groups with increasing influx. Initially, all groups 
are clearly distinct, groups 4-6 have relatively low occu- 
pation and the mean populations of ^2 and 5*3 are clearly 
above zero. As / grows while S2 and S3 lose population 
S4, S5 and 5*6 become more populated, but at different 
rates. Thus, at / « 150 the groups 5*4, 5*5 and Sq become 
united leading to the formation of a single set Sh of high 
mean occupancy vertices. Similarly, in the case of Si, S2, 
and 5*3. For / w 140 group 5*2 joins Si and becomes in- 
distinctive from 5*3 at 175 leading thus to a single 
low occupancy group Sl ■ During this process the overall 
mean population of Ft, (n), is growing. Consequently, 
for I € [175,220] again a 2-group pattern of 1124 high 
mean occupancy and 2972 low mean occupancy vertices 
appears, consisting now entirely of long living singletons 
'sustained by the influx'. Transition patterns before this 
pattern appears are 5- and 4-group patterns, after it grew 
unstable 3-group patterns are forming. 

As I is further increased the influx which heretofore 
guaranteed the sustenance of ^5 and Sg gradually causes 
overstimulation. Thus, the mean populations of 6*5 and 
Sq are shrinking, causing in turn less overstimulation to 
the core. So, again a small population can persist in 
S2 and S3. Altogether, for further increased influx the 
different groups disentangle, leading to a revival of the 
multi-group patterns. 




FIG. 23. Visualization of the network structure of the 
groups Si,...,Sa. The squares indicate many connections of 
vertices of a group among themselves, isolated circles in a 
group visualize that it consists of singletons. The thickness 
of the connecting hues gives a measure of the number of links 
connecting elements of different groups. Because of their im- 
portance for the existence of giant components in Ft the core 
groups S2 and S3 are bounded by fat lines. Larger boxes 
correspond to larger groups. 



more and more dominate and finally lead to disorder. 

Interestingly, the group structure allows some insight 
into the actual cluster structure of Ft at an arbitrary 
timestep. Obviously, giant and greater clusters on Ft 
can only emerge as long as the core is populated. As 
the mean population of S2 does not suffice to form a 
giant component within 52, ^3 functions as the 'glue' 
of giant clusters on Ff. Elements of the 'backbone' of 
the giant cluster preferably come from S2 and S'3. As 
the population of S2 decreases vertices of S^ increasingly 
become detached from the core and tend to form small 
clusters and singletons. 

A great component, however, is still retained by ele- 
ments of Sq and S2 bound together by S'3. As the popu- 
lation of S2 also sinks below a certain threshold, greater 
components of Ft become star-like, their hubs being in 
S'3. Finally clusters different from singletons completely 
vanish, resulting in purely influx sustained patterns of 
singletons. 



VII. PATTERNS FOR ti > 1 

For completeness in this section stationary patterns 
appearing for lower thresholds ti > 1 will be sketched 
briefly. 

Figure ^ displays data for the mean occupancy rates 
of every vertex gathered from a simulation on G^\^ with 
(ii,iu) = (3, 10). One reahzes two relatively broad bands 
of mean occupancy rates. Investigating the structures 
of Sh = S'(0.5) and Sl = G\Sh reveals that, analo- 
gously to the t\ < 1-situation both high- and low-mean 
occupancy groups completely decay into 2-clusters. Now, 
however, such 2-cluster patterns are only existing with 
mean occupancy rates {ajsn significantly smaller than 
1 and (a) Si far above zero. Thus, since a 'frozen' 2- 
clustered state (see Sec. Ill) can not subsist for tj > 1, 



a fluctuating 2-cluster pattern is formed. At every single 
timestep only a part of the high- mean occupancy vertices 
is occupied. As a consequence, not all holes are stable. 
A part of the holes can become occupied, thus giving 
the necessary stimulation to the vertices of Sh which are 
present. On average, whereas elements of Sl have high 
degrees (and are on the verge of being overstimulated) 
elements of Sh have low degrees (and are threatened by 
understimulation). On the whole, however, vertices occu- 
pying holes tend to get removed with higher probability. 
One can imagine this situation as a pattern of long living 
occupied 2-clusters flown round by short living 1- and 
2-clusters, living just for their stimulation. 
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FIG. 24. Relative occupation Si — i/\Si\'}2ves- ^(^) of the 
6 groups (labelled by their group number) for different in- 
fluxes. For I ~ 140 groups Si and S2 unite, for / « 175 
S3 joins them. At I ~ 220 S3 again becomes separated, 
for / « 240 also ^2 disjoins from Si. Similarly, S4, S5 
and Se, merge at / « 150. This leads to 5-group pat- 
terns (/ G [140,150]), 4-group patterns (/ £ [150,175] and 
I e [240,260]), singleton-patterns (/ G [175,220]) and a 
3-group pattern (/ G [220,240]). For I > 260 transitions 
between isomorphic configurations set in. 
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FIG. 25. The mean occupancy rate of every vertex from 
simulations on G^^^ with (ti, tu) = (3, 10) and influxes I = 150, 
300 and 500. For low influx both high-mean-occupancy group 
and low-mean-occupancy group are considerably apart from 
their extreme values 1 and 0. As / grows, Sl becomes less 
occupied and vertices of Sh evolve to almost permanent pres- 
ence. Thus, increasing I further the high mean occupancy set 
loses stimulation and the pattern becomes unstable. 



Considering this situation as evolved from an ideal 2- 
cluster pattern B(^g^ p^ one understands that it can only 
survive for an intermediate range of influxes. On the one 
hand, if the influx is too low, not enough occupied sites 
are driven out by overstimulation that holes could lose 
stability. Thus, the sites occupied in the ideal pattern B 
did not receive the necessary stimulation and the pattern 
was unstable. On the other hand, too high influx causes 
more and more of the frequently occupied vertices to be 
almost permanently occupied. This but leads to many 
stable holes. So, for higher influxes an abrupt transition 
to disorder occurs. The intermediate behaviour- tran- 
sitions between isomorphic configurations- is lacking in 
this case. 



VIII. CONCLUSIONS 

To conclude, we have presented a probabilistic model 
for a local-rule governed evolution of occupied and empty 
sites on regular graphs G. Aiming chiefly at a description 
of INW's in the immune system we studied two types 
of base graphs G'"'^' and G*^^^ created by describing id- 
iotypes by bit-strings and their functional interactions 
by 'matching-rules'. In contrast to most modelling ap- 
proaches to idiotypic networks ||^, the model abstains 
from all details of the dynamics of the real-life interac- 
tions of cells, but aims at understanding principal mech- 
anisms of network formation. 

On G'-^-' and G^^-* the dynamics generated by the win- 
dow algorithm leads to organized network structures, 
which consist of functionally different subsets, distin- 
guished, e.g., by their mean occupancy. On both types 
of base graphs for influxes / small compared to the sys- 
tem size, a multitude of such network-patterns has been 
found. We developed a notion of pattern-stability and 
classified them as metastable. For larger influxes one 
type of pattern prevails. In the case of the one-mismatch 
graphs G^^^ patterns for intermediate influx always con- 
sist of an arrangement of 2-clusters of holes and occupied 
vertices. We have given a classification of these patterns 
and have described fluctuations around an ideal pattern 
structure by a statistical approach via defects. For I > Ic 
these 2-clustered patterns become unstable, and the sys- 
tem starts oscillating between periods during which a 
pattern structure prevails and periods of disorder. Fi- 
nally, increasing / further, the dynamics is marked by 
randomness and extremely short lifetimes. We elucidated 
the nature of these transitions. 

These 2-clusters are nothing but the idiotype- 
antiidiotype pairs which have been proposed as one mech- 
anism for the preservation of memory of previously en- 
countered antigen (The antiidiotype represents the inter- 
nal image of the antigen [ pT]j3^ ) . Their arrangement in 
a coherent pattern, however, leads to an overall memory 
capacity of the network which increases only logarithmi- 
cally with the system size. This is clearly insufficient for 
a real immune system. The parameter range where the 
coherent 2-cluster patterns emerge can therefore not be 
the working regime of a healthy immune system. 

It is worthwhile to note that in this regime (and in 
all regimes allowing for memory on G(2)) the network 
connectivity decreases with growing simulation time, i.e. 
with growing age. This is in accord with experimen- 
tal observations and was also found in other modelling 
approaches [^j. It suggests that the networks gradually 
losing links is an essence of the limited range of allowed 
vertex-degrees and hence of cross-linking. 

On the more densely wired graphs G^^-* cube patterns 
have been found for an intermediate range of influxes. 
Increasing / beyond a threshold, the system then settles 
into dynamic 'multiple-group' patterns. In this regime, 
several subsets Sj of the base graph G^^' can be dis- 
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tinguished according to their mean occupancy. Vertices 
of these subsets also differ in the structure of the clus- 
ter on Ft which they typically belong to. Two of these 
subsets form a strongly interconnected core on Ff, to 
which vertices of the other groups are bound with dif- 
ferent strengthes. Each of the other subsets consists only 
of singletons and links to the core and some of the other 
groups. The core groups have been found to be always 
rarely populated, the major fraction of the population 
being typically contained in two of the other groups. 

The structure encountered in this parameter regime 
is in very good agreement with general ideas about the 
topology of INW's It exhibits a structured 

core which could correspond to a central part generally 
believed to exist in INW's. Furthermore, richely pop- 
ulated non-core groups are in good agreement with the 
notion of a peripheral part of INW's. Long lifetimes of 
vertices belonging to these groups are in accord with the 
idea that the peripheral part of the network is responsible 
for the preservation of idiotypic memory. 

The issue about the in detail- working of idiotypic mem- 
ory is still much disputed. Apart from the 'internal im- 
age hypothesis' several mechnisms are currently known 
pO|-^. One of these bases on certain cells (folicular 
dendritic cells) which occasionally exhibit parts of anti- 
gen, which is deemed enough to sufficiently stimulate the 
complementary idiotype - allowing in this way idiotypic 
memory in a single antigen-specific clone. 

Derived from our model a new mechanism for idiotypic 
memory seems possible: Memory could be retained in a 
single antigen-specific clone which is not preferably stim- 
ulated, but occasionally receives stimulation from the 
central part of the network and from new complemen- 
tary idiotypes from the bone marrow which serves as a 
background stimulus. This mechanism could collaborate 
with antigen presentation by follicular dendritic cells. 

Two further issues which follow from the multi-group 
structure of networks in our model seem worthwhile to 
note. 

First, during its evolution the system generates a group 
Si of always suppressed idiotypes. Although the dynam- 
ics of antigens differs from that of idiotypes, the existence 
of such a group implies that there should be a set of anti- 
gens against which an individual is immune without ever 
having been immunized against. The actual location of 
this set is determined by encounters with other antigens 
and, chiefly, by the fortuitous history of the deployment 
of bone marrow influx during early life. 

As a second fact, while the number of autoantibod- 
ies increases p4[ the bone marrow production is known 
to decrease with growing age Here, additional to 

a decline in connectivity till stationarity is reached, our 
model suggests another structural alteration of the net- 
work. Assuming for young individuals a working point 
of INW's which is above the parameter regime of single- 
clone patterns, a decreased bone marrow production 
leads to smaller cores and an increasing periphery, i.e. 



an accumulation of memory and loss of plasticity. 

Summarizing our main result, taking into account dif- 
ferent levels of coarse-graining of reaction affinities be- 
tween idiotypes -as represented by the base graphs G^^^ 

(2) 

and - leads to two different regimes of steady-state 
behaviours: (i) rigid 'coherent' configurations on G*-^-* 
and (ii) dynamic patterns on G*-^-* which are increasingly 
dominated by 1-clones. We conjecture that generally iso- 
lated clones are a consequence of a high connectivity of 
the network and high interaction strength between idio- 
types. It is suggestive that the bone marrow influx is 
a driving mechanism for shaping the structure of INW's, 
which together with the interaction strength of idiotypes, 
determines the working point of an INW. 
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